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This  report  is  structured  in  the  same  format  as  annual  performance  and  progress  reports.  It  has  the  same 
sections,  and  each  section  contains  relevant  cumulative  information  over  the  entire  project  period. 

1.  Personal  Information 

PI  Last  name  and  first  name: 

Performing  Institute: 

Mailing  Address: 

City: 

State: 

Zip  Code: 

Telephone: 

Fax: 

Email  Address: 

Your  website  URL: 

Contract  or  Grant  Number: 

Contract  or  Grant  Original  Title: 

ONR  Program  Officer: 

2.  Co-Principal  Investigator 

None 

3.  Progress  Summary 

3.1  Discussion  of  how  the  project  results  respond  to  the  objectives  of  ONR 

The  amount  of  information  that  military  decision  makers  and  planners  have  to  deal  with  and  its  rate  of 
change  are  both  increasing  rapidly,  as  the  scope  and  technologies  of  war  fighting  change.  For  instance, 
geographic  separation  between  the  command  and  control  center  (typically  located  in  the  US)  and  the 
actual  theatre  (which  could  be  anywhere  in  the  world),  availability  of  continuous  information  on  location 
and  movements  of  various  units,  real-time  weather  information,  and  increased  reliance  on  unmanned 
aerial,  land  and  underwater  reconnaissance  vehicles  have  all  necessitated  both  the  development  of 
information  visualization  techniques  and  the  use  of  large  information  displays.  An  example  of  such 
display  technology  is  the  Knowledge  Web/Wall  project  at  Navy’s  San  Diego  SPAWAR  Systems  Center. 

However,  the  physical  size  of  information  displays,  however  large,  is  still  limited.  Trying  to  present  too 
much  information  too  rapidly  on  a  fixed  size  display  can  result  in  information  overload,  stress  and  poor 
performance  on  the  part  of  the  operator.  One  solution  to  this  is  adaptively  and  progressively  providing 
information  on  the  display  while  guiding  the  user’s  attention.  While  this  seems  like  a  common  sense 
solution  to  the  information  overload  problem,  basic  research  that  illuminates  how  best  to  serially  present 
information,  how  to  guide  the  visual  attention  of  a  problem  solver  on  an  information  display,  and  how  to 
provide  details  at  the  right  time  and  right  place  on  the  display  to  achieve  optimal  performance  is  still 
lacking.  Without  such  research,  the  development  of  display  technology  alone  will  not  guarantee  that 
human  decision-makers  will  be  able  to  effectively  assimilate  and  act  on  the  information  being  presented. 
Results  from  this  project  contributed  to  advancing  the  knowledge  base  on  these  basic  research  issues. 
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The  work  applied  an  empirically  validated  cognitive  process  model  of  comprehension,  developed  in  prior 
ONR-supported  research,  to  generate  principled  ways  of  adaptively  presenting  information  to  users,  and 
measured  how  this  affected  problem-solving  performance.  The  primary  objectives  of  developing  adaptive 
information  presentation  techniques  based  on  a  theoretical  foundation,  and  empirically  demonstrating 
their  positive  influence  on  human  problem-solving  performance,  have  been  achieved.  In  particular,  we: 

>  developed  a  scientific  understanding  through  basic  experimental  research  of  how  a  cognitive 
model  could  be  applied  to  adaptive  information  display  design,  and  how  such  displays  affected 
human  problem-solving  performance;  and 

>  designed  human  interface  technology  for  adaptive  information  displays  that  tailored  presentations 
based  on  a  set  of  cognitive  principles  and  eye  movements  of  the  user. 

Enhancement  of  human  performance  has  been  designated  as  one  of  the  key  overarching  science  and 
technology  issues  for  the  Navy  of  the  future.  In  a  technology-dependent  Navy,  complex  information 
displays  will  be  omnipresent  and  superior  situational  awareness  and  quick-reaction  decision  making  will 
be  required  of  personnel.  Basic  scientific  research  on  how  to  improve  information  assimilation  and 
problem  solving  through  adaptive  displays  can  therefore  potentially  influence  the  design  of  future  Navy 
information  presentation  systems.  Such  research  can  also  aid  current  projects  such  as  the  Knowledge 
Web/Wall.  As  the  Navy  relies  more  and  more  on  remote-sensing  technologies  to  monitor  and  plan 
operations,  the  corresponding  information  displays  will  rely  heavily  on  graphical  representations.  Thus, 
the  capability  to  collect,  analyze  and  correlate  users’  gaze  patterns  on  graphical  displays  to 
comprehension  and  performance  measures  that  we  developed  in  this  project  should  prove  useful  to 
evaluate  the  efficacy  of  such  displays.  This  is  how  this  research  responds  to  ONR’s  Human  Interface 
Systems  Technology  objective  of  optimizing  the  interface  between  the  operator  and  equipment  for 
improved  efficiency.  This  research  matches  with  the  program  thrust  on  multimedia  displays,  visualization 
and  presentation.  It  also  has  relevance  to  the  thrusts  on  decision  support  and  workstation  interfaces. 

3.2  Project  tasks 

The  main  research  tasks  carried  out  are  enumerated  below. 

1  Theory  development  through  experimental  investigations  -  Further  developed  the  theoretical 
foundation  built  during  previous  ONR  supported  research  by  experimentally  investigating  how  a 
cognitive  process  model  of  multimodal  comprehension,  developed  in  that  prior  research  project,  could 
be  applied  to  improving  measurable  aspects  of  human  problem-solving  performance  from  information 
displays.  This  was  accomplished  by  experimentally  investigating  whether  and  to  what  extent  human 
performance  is  affected  when  an  information  display  adapts  its  presentation  according  to  each  of  the 
following  techniques  derived  from  the  cognitive  model: 

1.1  Compared  performance  when  a  user’s  visual  attention  is  guided  along  the  lines-of-action 
produced  by  causal  influences,  to  when  the  display  provides  no  such  guidance. 

1 .2  Compared  performance  when  the  display  shows  local  processes  through  visualization 
techniques  such  as  animation,  to  when  the  display  is  static. 

1 .3  Compared  performance  when  the  information  display  is  initially  sparse,  with  detailed 
information  being  progressively  revealed,  to  when  the  display  presents  all  information 
simultaneously. 

1 .4  Compared  performance  when  relevant  background  information  is  dynamically  displayed,  to 
when  the  display  requires  a  user  to  explicitly  look  at  a  static  presentation  of  background 
information. 
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2  Designed  technology  -  algorithms  and  implemented  software  -  for  adaptive  information  displays  that 
collects  information  regarding  the  user’s  gaze  and  applies  this  information,  in  accordance  with  the 
above  techniques,  to  modulate  information  presentation  in  ways  that  improved  human  performance. 

3  Trained  graduate  computer  science  students  in  cognitive  and  computational  aspects  of  Human 
Interface  Systems  Technology:  designing  and  implementing  advanced  interaction  techniques,  and 
designing,  conducting  and  analyzing  data  from  experiments  with  human  users. 

4  Dissemination  -  results  were  disseminated  through  publications  and  presentations  at  annual  ONR  PI 
meetings  and  other  venues. 


3.3  Description  of  reactive  information  display  designs  developed 


1  A  display  that  automatically  highlights  the  next  piece  of  information  a  user  must  attend  to  in  order  to 
solve  a  presented  causal  or  algorithmic  reasoning  problem  in  an  optimal  way,  in  order  to  draw  the 
user’s  attention  to  that  information  from  wherever  he  or  she  is  looking  currently,  and  does  this  for 
every  piece  of  relevant  information  on  the  display  in  an  optimal  order. 

2  A  display  that  automatically  highlights  the  current  piece  of  information  a  user  is  attending  to,  i.e.,  that 
tracks  and  responds  to  the  shifts  of  the  user’s  visual  attention. 

A  display  that  automatically  highlights  the  next  piece  of  information  a  user  must  attend  to  in  order  to 
solve  a  presented  causal  or  algorithmic  reasoning  problem  in  an  optimal  way,  where  this  next  item  is 
dynamically  selected  by  the  system  based  on  the  last  piece  of  information  the  user  attended  to  and 
dwelled  on  for  a  time  interval  above  a  threshold. 

4  A  display  that  automatically  highlights  the  next  piece  of  information  a  user  must  attend  to  in  order  to 
solve  a  presented  causal  or  algorithmic  reasoning  problem  in  an  optimal  way,  where  this  next  item  is 
dynamically  selected  by  the  system  based  on  the  problem  solving  procedure  the  user  needs  to  employ 
for  optimal  performance,  and  tracks  the  user’s  gaze  to  ensure  that  the  information  is  attended  to  and 
dwelled  on  for  a  time  interval  above  a  threshold. 

5  A  display  that  does  not  react  as  long  as  the  user  is  attending  to  pieces  of  infonnation  in  an  order 
conforming  to  the  problem  solving  procedure  the  user  needs  to  employ  for  optimal  performance,  but 
highlights  the  next  piece  of  information  to  be  attended  to  when  it  detects  that  the  user’s  gaze  breaks 
from  this  optimal  pattern  until  his  or  her  attention  is  drawn  back  to  the  appropriate  piece  of 
information. 

6  A  display  that  does  not  react  as  long  as  the  user  is  attending  to  pieces  of  infonnation  in  an  order 
conforming  to  the  problem  solving  procedure  the  user  needs  to  employ  for  optimal  performance,  but 
highlights  the  next  piece  of  information  to  be  attended  to  when  it  detects  that  the  user’s  gaze  breaks 
from  this  optimal  pattern  until  his  or  her  attention  is  drawn  back  to  the  appropriate  piece  of 
information,  or,  if  the  user  starts  looking  at  a  different  path  in  the  display,  employs  the  same 
highlighting  strategy  along  this  new  path. 

7  A  display  that  automatically  animates  the  dynamic  behavior  of  the  next  piece  of  information  (e.g.,  the 
rotation  of  a  mechanical  component)  a  user  must  attend  to  in  order  to  solve  a  presented  causal  or 
algorithmic  reasoning  problem  in  an  optimal  way,  in  order  to  draw  the  user’s  attention  to  that 
information  from  wherever  he  or  she  is  looking  currently,  and  does  this  for  every  piece  of  relevant 
information  on  the  display  in  an  optimal  order. 
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8  A  display  that  automatically  animates  the  dynamic  behavior  of  the  current  piece  of  information  (e.g., 
the  rotation  of  a  mechanical  component)  a  user  is  attending  to,  i.e.,  that  tracks  and  responds  to  the 
shifts  of  the  user’s  visual  attention. 

9  A  display  that  automatically  animates  the  dynamic  behavior  of  the  next  piece  of  information  a  user 
must  attend  to  in  order  to  solve  a  presented  causal  or  algorithmic  reasoning  problem  in  an  optimal 
way,  in  order  to  draw  the  user’s  attention  to  that  information  from  wherever  he  or  she  is  looking 
currently,  while  additional  background  information  is  shown  in  a  separate  region  of  the  display  in  a 
static  manner. 

10  A  display  that  automatically  animates  the  dynamic  behavior  of  the  next  piece  of  information  a  user 
must  attend  to  in  order  to  solve  a  presented  causal  or  algorithmic  reasoning  problem  in  an  optimal 
way,  in  order  to  draw  the  user’s  attention  to  that  information  from  wherever  he  or  she  is  looking 
currently,  while  showing  additional  textual  information  close  to  the  animated  information  in  a 
dynamic  fashion. 

1 1  A  display  that  starts  out  blank,  then  progressively  reveals  information,  one  item  at  a  time,  in  the  order 
that  a  user  must  attend  to  them  so  as  to  solve  a  presented  causal  or  algorithmic  reasoning  problem  in 
an  optimal  way. 


3.4  Summary  of  experiments  with  reactive  information  displays 


1  We  conducted  an  experiment  to  compare  hardware  and  software  based  eye-tracking  methods. 

2  We  conducted  a  second  experiment  to  investigate  the  strategies  of  successful  and  unsuccessful 
problems  solvers  in  tasks  requiring  them  to  solve  mechanical  reasoning  problems  from  diagrams. 

3  We  conducted  a  third  experiment  to  investigate  the  role  of  mental  imagery  in  solving  mechanical 
reasoning  problems  in  the  presence  and  absence  of  external  diagrams.  One  hundred  and  eleven 
engineering  students  participated  in  these  experiments  as  subjects. 

4  We  conducted  seven  more  experiments  involving  355  engineering  students  to  evaluate  nine  of  the 
above  described  reactive  information  displays. 


3.5  Summary  of  software  developed 

1  We  developed  software  that  labels  fixations  with  names  of  objects  on  the  display  so  that  raw  eye 
movement  data  can  be  reduced  to  a  temporal  series  of  display  objects  attended  to. 

2  We  developed  software  that  allows  an  experimenter  to  define  a  region  of  interest  on  a  display  by 
clicking  and  dragging  the  mouse,  and  then  automatically  computes  focus  shifts  in  and  out  of  the  region 
along  with  associated  fixation  times. 

3  In  addition  to  standard  measures  of  problem  solving  performance  such  as  time-on-task  and  accuracy, 
we  developed  two  measures  of  how  systematic  a  user’s  visual  search  on  a  display  is.  These  are  called 
coverage  and  order.  Coverage  is  defined  as  the  percentage  of  objects  in  the  display  that  were  attended 
to  for  more  than  a  time  interval  threshold.  It  is  a  number  between  0  and  100.  Order  is  a  measure  of 
how  closely  a  user’s  gaze  shifts  followed  the  optimal  order  of  attending  to  various  pieces  of 
information  on  the  display  as  dictated  by  an  optimal  problem  solving  procedure.  We  developed 
software  to  automatically  calculate  order  and  coverage  from  raw  eye  movement  data  and  information 
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about  boundaries  of  various  display  objects. 

4  We  developed  software-based  implementations  of  nine  of  the  reactive  information  displays  described 
above  so  that  a  computer  running  the  display  was  linked  to  another  computer  controlling  a  head- 
mounted  Eye  Link  eye  tracker. 

5  We  combined  the  various  pieces  of  eye  movement  data  reduction  and  analysis  software  to  produce  an 
integrated  eye  movement  data  analysis  tool. 

6  We  integrated  the  various  reactive  information  display  software  to  create  a  reactive  information 
display  design  toolkit  to  make  the  process  of  prototyping  and  evaluating  such  displays  easier  in  future. 

4.  Long  Term  Goals 

Our  long-term  goals,  beyond  the  end  of  the  ONR  project  period,  are  to  (1 )  invent,  implement  and  evaluate 
additional  reactive  information  displays  that  use  different  information  adaptation  strategies,  (2)  thereby 
develop  a  theoretical  foundation  for  the  principled  design  of  information  displays  based  on  tracking  and 
responding  to  user’s  attention  shifts  in  order  to  improve  problem  solving  performance,  and  (3)  to  apply 
our  techniques  to  realistic  decision  making  and  problem  solving  scenarios  in  the  domains  of  planning  and 
emergency  response. 

5.  Results:  Best  Accomplishments 

1  The  development  of  a  measure  of  how  closely  a  user’s  gaze  shifts  followed  the  optimal  order  of 
attending  to  various  pieces  of  information  on  the  display  as  dictated  by  an  optimal  problem  solving 
procedure.  An  efficient  and  successful  problem  solver  will  not  only  attend  to  all  relevant  objects  in  the 
display,  but  also  consider  them  in  the  order  that  best  supports  reasoning.  Therefore,  the  metric  that  we 
developed,  called  “order,”  is  one  that  measures  how  systematically  a  user  attended  to  causally  related 
elements  of  the  display.  It  is  defined  as  follows.  Let  S  be  an  ordered  sequence  of  display  objects  that  a 
user  attended  to  during  a  problem  solving  session.  So  S  begins  with  the  first  display  item  attended  to, 
and  ends  with  the  last  item  attended  to  before  the  solution  to  the  problem  is  produced.  This  sequence  is 
generated  from  eye  movement  data.  In  this  sequence,  if  object  j  appears  immediately  after  object  i,  and 
if  i  can  causally  influence  j  according  to  the  lines  of  action  in  the  system,  then  i-j  represents  a  causal 
link  in  the  sequence  S.  Consecutive  causal  links  represent  causal  subsequences  of  S.  The  length  of  a 
causal  subsequence  is  the  number  of  causal  links  in  it.  Order  of  S  is  defined  as  the  sum  of  squares  of 
the  lengths  of  causal  subsequences  in  S.  This  captures  the  correctness  of  the  order  in  which  the  user 
processed  the  display  (i.e.  each  causal  link  indicates  that  the  problem  solver  considered  one  accurate 
cause-effect  pair  of  display  objects)  weighted  by  the  length  of  lines  of  action  that  have  been 
considered  (i.e.  if  users  A  and  B  both  considered  the  same  number  of  causal  links,  but  if  A  looked  at 
longer  chains  of  causal  links  than  B,  the  value  of  order  will  be  higher  for  A  than  B).  Order  is  an 
integer  greater  than  or  equal  to  zero. 

2  Experimentally  demonstrated  that  while  a  hardware  based  eye  tracker  is  faster  and  more  natural  to  use, 
data  accuracy  is  higher  for  a  software  based  eye  tracker  operating  on  a  “key-hole  view”  principle. 

3  Experimentally  characterized  aspects  of  successful  causal  reasoning  and  problem  solving:  longer  time- 
on-task  and  attending  to  more  components  of  the  problem  do  not  necessarily  lead  to  success  in 
mechanical  reasoning  problems  presented  on  graphical  displays;  instead,  systematically  searching  the 
display  with  higher  values  of  the  measure  order  (described  above)  and  concentrating  on  critical 
components  of  the  problem  lead  to  higher  accuracy. 
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4  We  did  not  find  a  significant  difference  in  accuracy  between  those  who  employed  mental  imagery  in 
solving  problems  of  the  kind  described  above  and  those  who  did  not.  However,  successful  problem 
solvers  who  employed  mental  imagery  were  more  systematic  in  their  allocation  of  visual  attention,  and 
concentrated  more  on  critical  components,  than  unsuccessful  problem  solvers  who  employed  mental 
imagery. 

5  We  found  that  reactive  information  displays  encouraged  systematic  visual  search  and  improved 
problem  solving  performance.  See  section  8  for  details. 


6.  Scientific  and  Technical  Objectives 

The  main  objectives  of  the  research  effort  were  to  (i)  develop,  through  basic  experimental  research,  a 
cognitive  approach  that  can  inform  display  designers  as  to  how  information  presentations  may  be  adapted 
to  improve  human  problem-solving  performance,  and  (ii)  develop  and  empirically  evaluate  several 
technical  designs  for  such  displays  for  different  problem  types  and  domains. 

7.  Approach 

Our  approach  consisted  of  the  following  steps: 

1.  Invent  approaches  to  adapt  information  displays  in  real  time  based  on  eye  movement  data  and 
principles  derived  from  a  cognitive  model  of  information  comprehension  developed  in  an  earlier 
ONR  project. 

2.  Design  corresponding  reactive  information  displays  for  presenting  problems  in  the  mechanical 
and  algorithmic  domains  to  users  to  solve. 

3.  Implement  display  prototypes. 

4.  Experimentally  evaluate  the  displays. 

5.  Use  the  findings  to  develop  a  theory  of  reactive  information  displays. 

8.  Impact  of  Basic  Research 

This  research  made  several  contributions.  The  primary  one  was  a  theoretical  framework  for  the  principled 
design  and  evaluation  of  information  displays  that  track  the  visual  attention  of  users  and  react  according 
to  a  generalized  model  of  comprehension  and  problem  solving.  Four  research  questions  about  the  impact 
of  reactivity  were  raised  and  investigated  —  Is  problem  solving  performance  improved  when:  (1)  The 
user’s  visual  attention  is  guided  along  paths  of  causal  influence?  (2)  Local  component  behaviors  are 
animated  in  regions  of  the  display,  after  the  user’s  attention  has  been  attracted  to  the  regions?  (3)  The 
information  display  is  initially  sparse,  with  detailed  information  being  progressively  revealed?  (4) 
Relevant  background  information  is  shown  in  regions  of  the  display  while  the  user  attends  to  those 
regions? 

To  examine  the  efficacy  of  reactive  information  displays  implementing  these  strategies,  we  measured 
nine  dependent  variables:  accuracy  of  problem  solving,  post-test  score,  response  time,  coverage,  number 
of  focus  shifts  between  critical  elements  of  the  display,  number  of  focus  shifts  between  all  display 
elements,  order  of  visual  attention  shifts,  dwell  time  on  critical  components,  and  animation  tracking 
behavior  of  users. 
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The  following  are  our  findings.  Users  with  higher  accuracy  also  score  higher  in  the  post-test.  Users  who 
take  longer  to  solve  a  problem  also  tend  to  exhibit  higher  coverage,  order  and  number  of  focus  shifts. 
Reactive  information  displays  with  animations  that  guide  a  user’s  visual  attention  along  causal  paths 
increased  accuracy  and  also  increased  coverage,  number  of  focus  shifts,  dwell  time  on  critical 
components,  order  (i.e.  systematicity  of  visual  search)  and  response  time.  Reactive  information  displays 
that  progressively  revealed  information  increased  accuracy,  order  and  critical  component  dwell  time 
while  decreasing  coverage.  Reactive  information  displays  that  guided  a  user’s  visual  attention  to  the  next 
display  element  according  to  an  optimal  search  strategy,  or  followed  a  user’s  gaze  patterns  to  detect 
deviation  from  this  strategy  and  upon  such  detection  highlighted  relevant  portions  of  the  display  to  get  the 
user  back  on  track,  improved  accuracy,  coverage,  order  of  visual  search,  critical  component  dwell  time 
and  post-test  scores.  Reactive  information  displays  with  animations,  progressive  revealing  and  next 
component  highlighting  improved  accuracy,  and  increased  coverage,  focus  shifts,  order,  critical 
component  dwell  time,  response  time  and  post-test  score.  In  summary,  we  have  strong  initial  evidence 
that  reactive  information  displays  positively  influence  multiple  aspects  of  a  problem  solver’s  visual  search 
and  accuracy. 

This  project  has  thus  contributed  to  the  development  of  a  cognitive  comprehension  model  and  a 
principled  approach  to  the  design  of  information  displays  that  adapt  information  presented  in  response  to 
user’s  gaze  patterns  and  in  accordance  with  known  optimal  procedures  for  problem  solving.  These  results 
have  the  potential  to  improve  the  design  of  large-scale  information  displays  that  military  commanders  and 
emergency  response  centers  use  to  provide  information  in  real  time  to  decision  makers  and  planners.  The 
eye-tracking  laboratory  at  Auburn  is  one  of  the  very  few  research  groups  in  the  United  States 
investigating  the  application  of  eye  tracking  to  information  display  design.  We  have  developed  various 
pieces  of  software  for  making  analysis  of  eye  movements  easier,  and  for  realizing  gaze-contingent 
displays.  These  technologies  have  mainly  been  used  in-house  for  supporting  basic  research.  Since  the 
focus  of  this  research  project  was  not  developing  and  disseminating  eye  tracking  technology  per  se,  the 
impact  of  the  technology  we  have  developed  is  indirect  in  terms  of  its  support  of  basic  research.  In  the 
longer  term,  we  expect  this  software  technology  development  effort  to  promote  more  widespread  use  of 
eye  movement  data  collection  and  analysis  in  basic  research  on  problem  solving,  planning,  decision¬ 
making,  etc. 

9.  Technology  Transfer 

Our  main  means  of  facilitating  technology  transfer  has  been  through  publications  and  presentations  at 
forums  attended  by  scientists  and  engineers  from  academia,  industry  and  military.  We  have  disseminated 
the  results  of  our  research  to  the  cognitive  psychology,  human-computer  interaction,  multimedia  and 
interactive  system  design  communities.  For  more  information,  see  sections  13  (publications)  and  13 
(presentations)  below. 

10.  Statistics 

Number  of  Degrees  Granted:  2 
PI/CoPI  Minority  Women**:  0 
PI/CoPI  Non-Minority  Women:  0 
PI/CoPI  Minority  Men**:  0 
PI/CoPI  Non-Minority  Men:  1 
Grad  Students  Minority  Women**:  0 
Grad  Students  Non-Minority  Women:  0 
Grad  Students  Minority  Men**:  0 
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Grad  Students  Non-Minority  Men:  2 

Post  Doctoral  Minority  Women**:  0 

Post  Doctoral  Non-Minority  Women:  0 

Post  Doctoral  Minority  Men**:  0 

Post  Doctoral  Non-Minority  Men:  0 

Undergrad  Students  Minority  Women**:  0 

Undergrad  Students  Non-Minority  Women:  0 

Undergrad  Students  Minority  Men**:  0 

Undergrad  Students  Non-Minority  Men:  0 

Number  of  journal  articles  in  preparation,  revision  or  review:  3 

Number  of  journal  articles  published:  0 

Number  of  articles  published  in  conference  proceedings  or  books:  5 
Number  of  technical  and  final  reports  authored:  2 
Number  of  patents  issued  or  pending:  0 

11  References  to  publications  resulting  from  this  project 

Note:  Several  journal  publications  are  in  various  stages  of  preparation,  revision  or  review.  Only  accepted  or 
published  articles  are  listed  below.  Copies  of  these  papers  (and  any  future  publications  arising  from  this  project) 
may  be  downloaded  from  http://www.eng.aubum.edu/~naraynh/pubs.html. 

1  Narayanan,  N.  H.  (2006).  Cognitively  based  design  of  expository  multimedia  on  causal  and  dynamic 
systems.  Chapter  in  Alamargot,  Terrier  &  Cellier  (Eds.),  The  Handbook  of  Production, 

Comprehension  and  Use  of  Technical  Documents  at  Work,  Elsevier  Science  Publishers,  to  appear. 

2  Yoon,  D.  &  Narayanan,  N.  H.,  Lee,  S.  &  Kwon,  O.  (2006).  Exploring  the  effect  of  animation  and 
progressive  revealing  on  diagrammatic  problem  solving,  Diagrammatic  Representation  &  Inference: 
Proceedings  of  the  Fourth  International  Conference  on  the  Theory  and  Application  of  Diagrams, 
Lecture  Notes  in  Artificial  Intelligence,  Vol.  LNAI  4045,  Springer,  pp.  226-240. 

3  Yoon,  D.  &  Narayanan,  N.  H.  (2004).  Mental  imagery  in  problem  solving:  An  eye  tracking  study. 
Proceedings  of  the  Third  ACM  Symposium  on  Eye  Tracking  Research  &  Applications,  Association  for 
Computing  Machinery  (ACM)  Press,  pp.  77-83. 

4  Yoon,  D.  &  Narayanan,  N.  H.  (2004).  Predictors  of  success  in  diagrammatic  problem  solving. 
Diagrammatic  Representation  &  Inference:  Proceedings  of  the  Third  International  Conference  on  the 
Theory  and  Application  of  Diagrams,  Lecture  Notes  in  Artificial  Intelligence,  Vol.  LNAI  2980, 
Springer-Verlag,  pp.  301-315. 

5  Narayanan,  N.  H.  &  Yoon,  D.  (2003).  Reactive  information  displays.  Proceedings  of  INTERACT 
2003:  Ninth  IFIP  TC 13  International  Conference  on  Human-Computer  Interaction,  IOS  Press,  pp. 
244-251. 

6  Yoon,  D.  &  Narayanan,  N.  H.  (2003).  What  separates  successful  and  unsuccessful  problem  solvers  in 
diagrammatic  reasoning?  Technical  Report  CSSE03-05,  Computer  Science  &  Software  Engineering 
Dept.,  Auburn  University,  Auburn,  AL  36849,  6  pages. 

12.  Honors 

1.  Narayanan  was  awarded  Auburn  University  College  of  Engineering  Senior  Faculty  Research 
Excellence  Award  in  2005,  in  part  recognizing  his  research  on  this  project. 
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13.  Presentations 


1.  Invited  Seminar,  "Understanding  and  reasoning  about  complex  systems:  A  cognitive  model  and 
its  application  to  multimedia  design,"  Electronics  &  Telecommunications  Research  Institute, 
Daejeon,  Korea,  October  2006. 

2.  Invited  Seminar,  “Reactive  information  displays:  Theory  and  applications,”  Electronics  & 
Telecommunications  Research  Institute,  Daejeon,  Korea,  October  2006. 

3.  Paper  Presentation,  “Exploring  the  effect  of  animation  and  progressive  revealing  on  diagrammatic 
problem  solving,”  Fourth  International  Conference  on  the  Theory  and  Application  of  Diagrams, 
Stanford  University,  June  2006. 

4.  Paper  Presentation,  "Contextualizing  learning  about  complex  systems  from  visualizations: 
cognitive  theory,  empirical  evidence  and  design  guidelines",  Annual  Meeting  of  the  American 
Educational  Research  Association  (AERA),  Montreal,  Canada,  April  2005. 

5.  Research  Seminar,  "Causal  learning  about  complex  systems,"  Florida  Institute  for  Human  and 
Machine  Cognition,  Pensacola,  FL,  March  2005. 

6.  Paper  Presentation,  "Reactive  information  displays:  A  progress  report,"  Office  of  Naval  Research 
Workshop  on  Attention,  Perception  and  Modeling  for  Complex  Displays,  Naval  Undersea 
Warfare  Center  Division,  Newport,  RI,  May  2004. 

7.  Paper  Presentation,  “Mental  imagery  in  problem  solving:  An  eye  tracking  study,”  Third  ACM 
Symposium  on  Eye  Tracking  Research  &  Applications,  San  Antonio,  TX,  March  2004. 

8.  Paper  Presentation,  “Predictors  of  success  in  diagrammatic  problem  solving,”  Third  International 
Conference  on  the  Theory  and  Application  of  Diagrams,  Cambridge,  UK,  March  2004. 

9.  Paper  Presentation,  “Reactive  information  displays,”  INTERACT  2003:  Ninth  IFIP  TC  13 
International  Conference  on  Human-Computer  Interaction,  Zurich,  Switzerland,  September  2003. 

10.  Paper  Presentation,  "Reactive  information  displays,"  Office  of  Naval  Research  Workshop  on 
Attention,  Perception  and  Data  Visualization,  Rensselaer  Polytechnic  Institute,  Troy,  NY,  June 
2003. 

14.  Related  Projects  and  Grants 

11.  Learning  about  causal  systems  in  complex  domains:  A  multidisciplinary  synthesis  of  the  state-of- 
the-art  and  research  challenges,  2003-2007,  National  Science  Foundation,  SBE0350342. 

1.  Analysis  and  Synthesis  of  Hypermedia  Information  Visualizations,  1999-2002,  Office  of  Naval 
Research,  N000 1 4961 1 187. 

2.  Participant  Support  for  Second  International  Conference  on  Theory  and  Application  of  Diagrams, 
2002,  Office  of  Naval  Research;  N000140210350. 

3.  Eye  Tracking  Equipment  for  Analysis  and  Synthesis  of  Human  Interactions  with  Hypermedia 
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Information  Visualizations,  2000-2001,  Defense  Universities  Research  Instrumentation  Program 
(DURIP)  administered  through  Office  of  Naval  Research,  N000140010418. 

4.  Equipment  for  Eye  Tracking,  1996,  this  equipment  was  provided  to  Auburn  University  by 
Camegie-Mellon  University  through  an  ONR-managed  research  equipment  grant  with  PI  John 
Anderson,  Department  of  Psychology,  Carnegie  Mellon  University. 
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